Decoding visual object categories in early somatosensory cortex

Neurons, even in the earliest sensory areas of cortex, are subject to a great deal of contextual influence from both within and across modality connections. In the present work, we investigated whether the earliest regions of somatosensory cortex (S1 and S2) would contain content-specific information about visual object categories. We reasoned that this might be possible due to the associations formed through experience that link different sensory aspects of a given object. Participants were presented with visual images of different object categories in 2 fMRI experiments. Multivariate pattern analysis revealed reliable decoding of familiar visual object category in bilateral S1 (i.e., postcentral gyri) and right S2. We further show that this decoding is observed for familiar but not unfamiliar visual objects in S1. In addition, whole-brain searchlight decoding analyses revealed several areas in the parietal lobe that could mediate the observed context effects between vision and somatosensation. These results demonstrate that even the first cortical stages of somatosensory processing carry information about the category of visually presented familiar objects

Innovative machine vision technique for 2D/3Dcomplex and irregular surfaces modelling

Abstract—This study propose and demonstrates a novel technique incorporating multilayer perceptron (MLP) neural networks for feature extraction with Photometric stereo based image capture techniques for the analysis of complex and irregular 2D profiles and 3D surfaces. In order to develop the method and to ensure that it is capable of modelling non-axisymmetric and complex 2D/3D profiles, the network was initially trained and tested on 2D profiles, and subsequently using objects consisting of between 1 and 4 hemispherical 3D forms. To test the capability of the proposed model, random noise was added to 2D profiles.3D objects were coated with various degrees of coarsenesses(ranging from low-high). The gradient of each surface normalwas quantified in terms of the slant and tilt angles of the vector about the x and y axis respectively. The slant and tilt angles were obtained from the bump maps and these data were subsequently employed for training of a NN that had x and y as inputs and slant and tilt angles as outputs. The network employed had the following architecture: MLP and a Levenberg-Marquardt algorithm (LMA) for training the network for 12,000 epochs. At each point on the surface the network was consulted to predict slant and tilt and the actual slant and tilt was subtracted, giving a measure of surface irregularity. The network was able to model the underlying asymmetrical geometry with an accuracyregression analysis R-value of 0.93 for a single 3D hemispheres and 0.90 for four adjacent 3D non-axisymmetric hemispheres

BRDF of human skin in the visible spectrum

© Emerald Publishing Limited. Purpose - Significant research has been carried out in terms of development of new bidirectional reflectance distribution function (BRDF) instruments; however, there is still little research available regarding spectral BRDF measurements of human skin. This study aims to investigate the variation in human skin reflectance using a new fibre optic-based spectral-BRDF measurement device. Design/methodology/approach - Design of this system mainly involves use of multiple fibre optics to illuminate and detect light reflected from a sample, whereas a hemispherical dome was 3D printed to mount the fibres at various slant/tilt angles. To investigate the spectral differences in BRDF of human skin, 3 narrowband filters in the visible spectrum were used, whereas measurements were taken from the back of the hand for Caucasian and Asian skin types. Findings - The experiments demonstrate that the BRDF of human skin varies with wavelengths in the visible spectrum and it is also different for Caucasian and Asian skin types. Both skin types exhibit off-specular reflection with increase in angle of incidence and show less variation with respect to viewing angles when the angle of incidence is normal to the surface. Research implications - A database of spectral BRDF measurements of human skin will help not only in creating realistic skin renderings but also in development of novel skin reflectance models for biomedical and machine vision applications. The measurements would also provide means to validate the predictions from existing light transport/spectral simulation models for human skin and will ultimately help in the accurate diagnosis and simulation of various skin disorders. Originality/value - The proposed system provides fast scatter measurements by utilising multiple fibres to detect light simultaneously at different angles while also allowing easy switching between incident light directions. Due to its flexible design and contact-based measurements, the device is independent of errors due to sample movements and does not require any image registration. Also, measurements taken from the device show that the BRDF of skin varies significantly in the visible spectrum and it is different for Caucasian and Asian skin types

An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region

© 2013 Elsevier B.V. All rights reserved. Although photometric stereo offers an attractive technique for acquiring 3D data using low-cost equipment, inherent limitations in the methodology have served to limit its practical application, particularly in measurement or metrology tasks. Here we address this issue. Traditional photometric stereo assumes that lighting directions at every pixel are the same, which is not usually the case in real applications, and especially where the size of object being observed is comparable to the working distance. Such imperfections of the illumination may make the subsequent reconstruction procedures used to obtain the 3D shape of the scene prone to low frequency geometric distortion and systematic error (bias). Also, the 3D reconstruction of the object results in a geometric shape with an unknown scale. To overcome these problems a novel method of estimating the distance of the object from the camera is developed, which employs photometric stereo images without using other additional imaging modality. The method firstly identifies Lambertian diffused maxima region to calculate the object distance from the camera, from which the corrected per-pixel light vector is able to be derived and the absolute dimensions of the object can be subsequently estimated. We also propose a new calibration process to allow a dynamic(as an object moves in the field of view) calculation of light vectors for each pixel with little additional computation cost. Experiments performed on synthetic as well as real data demonstrates that the proposed approach offers improved performance, achieving a reduction in the estimated surface normal error of up to 45% as well as mean height error of reconstructed surface of up to 6 mm. In addition, when compared to traditional photometric stereo, the proposed method reduces the mean angular and height error so that it is low, constant and independent of the position of the object placement within a normal working range

Obtaining malignant melanoma indicators through statistical analysis of 3D skin surface disruptions

Background/purpose: It has been observed that disruptions in skin patterns are larger for malignant melanoma (MM) than benign lesions. In order to extend the classification results achieved for 2D skin patterns, this work intends to investigate the feasibility of lesion classification using 3D skin surface texture, in the form of surface normals acquired from a previously built six-light photometric stereo device. Material and methods: The proposed approach seeks to separate MM from benign lesions through analysis of the degree of surface disruptions in the tilt and slant direction of surface normals, so called skin tilt pattern and skin slant pattern. A 2D Gaussian function is used to simulate a normal region of skin for comparison with a lesion's observed tilt and slant patterns. The differences associated with the two patterns are estimated as the disruptions in the tilt and slant pattern respectively for lesion classification. Results: Preliminary studies on11 MMs and 28 benign lesions have given Receiver operating characteristic areas of 0.73 and 0.85 for tilt and slant pattern, respectively, which are better than 0.65 previously obtained for the skin line direction using the same samples. Conclusions: This paper has demonstrated an important application of 3D skin texture for computer-assisted diagnosis of MM in vivo. By taking advantage of the extra dimensional information, preliminary studies suggest that some improvements over the existing 2D skin line pattern approach for the differentiation between MM and benign lesions. © 2009 John Wiley & Sons A/S

Gender recognition from facial images: Two or three dimensions?

© 2016 Optical Society of America. This paper seeks to compare encoded features from both two-dimensional (2D) and three-dimensional (3D) face images in order to achieve automatic gender recognition with high accuracy and robustness. The Fisher vector encoding method is employed to produce 2D, 3D, and fused features with escalated discriminative power. For 3D face analysis, a two-source photometric stereo (PS) method is introduced that enables 3D surface reconstructions with accurate details as well as desirable efficiency. Moreover, a 2D + 3D imaging device, taking the two-source PS method as its core, has been developed that can simultaneously gather color images for 2D evaluations and PS images for 3D analysis. This system inherits the superior reconstruction accuracy from the standard (three or more light) PS method but simplifies the reconstruction algorithm as well as the hardware design by only requiring two light sources. It also offers great potential for facilitating human computer interaction by being accurate, cheap, efficient, and nonintrusive. Ten types of low-level 2D and 3D features have been experimented with and encoded for Fisher vector gender recognition. Evaluations of the Fisher vector encoding method have been performed on the FERET database, Color FERET database, LFW database, and FRGCv2 database, yielding 97.7%, 98.0%, 92.5%, and 96.7% accuracy, respectively. In addition, the comparison of 2D and 3D features has been drawn from a self-collected dataset, which is constructed with the aid of the 2D + 3D imaging device in a series of data capture experiments. With a variety of experiments and evaluations, it can be proved that the Fisher vector encoding method outperforms most state-of-the-art gender recognition methods. It has also been observed that 3D features reconstructed by the two-source PS method are able to further boost the Fisher vector gender recognition performance, i.e., up to a 6% increase on the self-collected database

Photometric stereo for three-dimensional leaf venation extraction

© 2018 Elsevier B.V. Leaf venation extraction studies have been strongly discouraged by considerable challenges posed by venation architectures that are complex, diverse and subtle. Additionally, unpredictable local leaf curvatures, undesirable ambient illuminations, and abnormal conditions of leaves may coexist with other complications. While leaf venation extraction has high potential for assisting with plant phenotyping, speciation and modelling, its investigations to date have been confined to colour image acquisition and processing which are commonly confounded by the aforementioned biotic and abiotic variations. To bridge the gaps in this area, we have designed a 3D imaging system for leaf venation extraction, which can overcome dark or bright ambient illumination and can allow for 3D data reconstruction in high resolution. We further propose a novel leaf venation extraction algorithm that can obtain illumination-independent surface normal features by performing Photometric Stereo reconstruction as well as local shape measures by fusing the decoupled shape index and curvedness features. In addition, this algorithm can determine venation polarity – whether veins are raised above or recessed into a leaf. Tests on both sides of different leaf species with varied venation architectures show that the proposed method is accurate in extracting the primary, secondary and even tertiary veins. It also proves to be robust against leaf diseases which can cause dramatic changes in colour. The effectiveness of this algorithm in determining venation polarity is verified by it correctly recognising raised or recessed veins in nine different experiments

Innovative 3D and 2D machine vision methods for analysis of plants and crops in the field

© 2018 Elsevier B.V. Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light – particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all the associated cost and ecological benefits) as well as offering new capabilities for plant phenotyping. The considerable challenges associated with robust and reliable utilisation of machine vision in the field are also considered and practical solutions are described. Two projects are used to illustrate the proposed approaches: a four-light photometric stereo apparatus able to recover plant textures at high-resolution (even in direct sunlight), and a 3D system able to measure potato sizes in-the-field to an accuracy of within 10%, for extended periods and in a range of environmental conditions. The potential benefits of the proposed 3D methods are discussed, both in terms of the advanced capabilities attainable and the widespread potential uptake facilitated by their low cost

Eye centre localisation: An unsupervised modular approach

© Emerald Group Publishing Limited. Purpose - This paper aims to introduce an unsupervised modular approach for eye centre localisation in images and videos following a coarse-to-fine, global-to-regional scheme. The design of the algorithm aims at excellent accuracy, robustness and real-time performance for use in real-world applications. Design/methodology/approach - A modular approach has been designed that makes use of isophote and gradient features to estimate eye centre locations. This approach embraces two main modalities that progressively reduce global facial features to local levels for more precise inspections. A novel selective oriented gradient (SOG) filter has been specifically designed to remove strong gradients from eyebrows, eye corners and self-shadows, which sabotage most eye centre localisation methods. The proposed algorithm, tested on the BioID database, has shown superior accuracy. Findings - The eye centre localisation algorithm has been compared with 11 other methods on the BioID database and six other methods on the GI4E database. The proposed algorithm has outperformed all the other algorithms in comparison in terms of localisation accuracy while exhibiting excellent real-time performance. This method is also inherently robust against head poses, partial eye occlusions and shadows. Originality/value - The eye centre localisation method uses two mutually complementary modalities as a novel, fast, accurate and robust approach. In addition, other than assisting eye centre localisation, the SOG filter is able to resolve general tasks regarding the detection of curved shapes. From an applied point of view, the proposed method has great potentials in benefiting a wide range of real-world human-computer interaction (HCI) applications

Magnetic field waves at Uranus

The proposed research efforts funded by the UDAP grant to the BRI involve the study of magnetic field waves associated with the Uranian bow shock. This is a collaborative venture bringing together investigators at the BRI, Southwest Research Institute (SwRI), and Goddard Space Flight Center (GSFC). In addition, other collaborations have been formed with investigators granted UDAP funds for similar studies and with investigators affiliated with other Voyager experiments. These investigations and the corresponding collaborations are included in the report. The proposed effort as originally conceived included an examination of waves downstream from the shock within the magnetosheath. However, the observations of unexpected complexity and diversity within the upstream region have necessitated that we confine our efforts to those observations recorded upstream of the bow shock on the inbound and outbound legs of the encounter by the Voyager 2 spacecraft